{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ade08c8e",
   "metadata": {},
   "source": [
    "# Comp3-2组学(模态)融合 - Stacking后融合\n",
    "\n",
    "融合一般指的是对一个研究对象使用多种不同的数据，或者相同数据不同模型预测结果刻画结果的综合考虑。一般情况下，融合分为前融合、后融合\n",
    "\n",
    "* 前融合一般是数据层面的融合。\n",
    "* 后融合一般是结果层面的融合。\n",
    "\n",
    "我们分别使用Comp1-1抽取出来的组学特征与蛋白质组学进行前融合，使用`组学预测结果`与`量表数据`进行后融合。后偶融合分为两种：\n",
    "1. ensemble, 一般是最终结果进行融合，融合可以分为软投票和硬投票。\n",
    "2. stacking，一般是结果在使用一个机器学习算法模型进行融合。\n",
    "\n",
    "##### **注意：由于使用后融合技术，需要保持数据筛选的样本一样**\n",
    "\n",
    "将Comp1分析的组学数据的结果作为量表数据研究的特征，再使用一个特征进行融合。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a51aa47",
   "metadata": {},
   "source": [
    "## 一、数据校验\n",
    "首先需要检查诊断数据，如果显示`检查通过！`择可以正常运行之后的，否则请根据提示调整数据。\n",
    "\n",
    "当默认使用`pandas.read_csv`进行数据去取，然也可以使用自定义的函数，获取解析数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a24b9d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['KMP_DUPLICATE_LIB_OK'] = 'TRUE'\n",
    "import pandas as pd\n",
    "from pathlib import Path\n",
    "from onekey_algo.custom.components.Radiology import diagnose_3d_image_mask_settings, get_image_mask_from_dir\n",
    "from onekey_algo import OnekeyDS as okds\n",
    "\n",
    "# 设置数据目录\n",
    "# mydir = r'你自己数据的路径'\n",
    "mydir = okds.grade\n",
    "rad_trainf = '../comp1-传统组学/SVM_train.csv'\n",
    "rad_testf = '../comp1-传统组学/SVM_test.csv'\n",
    "\n",
    "# 读取标签数据列名\n",
    "labels = ['B超诊断阳性=1']\n",
    "\n",
    "meta_features = pd.read_csv(mydir, header=0)\n",
    "rad_train = pd.read_csv(rad_trainf, header=0)\n",
    "rad_test = pd.read_csv(rad_testf, header=0)\n",
    "\n",
    "print(f'分别获取到 meta: {meta_features.shape[0]}个样本')\n",
    "meta_features.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59b1e6c1",
   "metadata": {},
   "source": [
    "## 二、获取meta数据划分\n",
    "\n",
    "由于训练集和测试集在Rad上划分好，所以需要根据组学的划分，对meta进行划分。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ce7bc75",
   "metadata": {},
   "outputs": [],
   "source": [
    "meta_train = pd.merge(rad_train, meta_features, on=['ID'], how='inner')\n",
    "meta_test = pd.merge(rad_test, meta_features, on=['ID'], how='inner')\n",
    "\n",
    "# Drop掉ID\n",
    "meta_train = meta_train.drop(['ID'], axis=1)\n",
    "meta_test = meta_test.drop(['ID'], axis=1)\n",
    "\n",
    "meta_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c0bdb14",
   "metadata": {},
   "source": [
    "### 标注拆分"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afb20f03",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import onekey_algo.custom.components as okcomp\n",
    "import onekey_algo.custom.components.comp1 as comp1\n",
    "\n",
    "n_classes = 2\n",
    "\n",
    "# 生成训练数据集\n",
    "y_train_sel = meta_train[labels]\n",
    "X_train_sel = meta_train.drop(labels, axis=1)\n",
    "# 生成测试数据集\n",
    "y_test_sel = meta_test[labels]\n",
    "X_test_sel = meta_test.drop(labels, axis=1)\n",
    "\n",
    "print(f\"训练集样本数：{X_train_sel.shape}, 验证集样本数：{X_test_sel.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "011d4d1d",
   "metadata": {},
   "source": [
    "## 模型筛选\n",
    "\n",
    "根据筛选出来的数据，做模型的初步选择。当前主要使用到的是Onekey中的\n",
    "\n",
    "1. SVM，支持向量机，引用参考。\n",
    "2. KNN，K紧邻，引用参考。\n",
    "3. Decision Tree，决策树，引用参考。\n",
    "4. Random Forests, 随机森林，引用参考。\n",
    "5. XGBoost, bosting方法。引用参考。\n",
    "6. LightGBM, bosting方法，引用参考。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e143f98",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_names = ['SVM', 'KNN', 'DecisionTree', 'RandomForest', 'ExtraTrees']\n",
    "models = comp1.create_clf_model(model_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3d5398e",
   "metadata": {},
   "source": [
    "## 模型筛选\n",
    "\n",
    "使用最好的数据划分，进行后续的模型研究。\n",
    "\n",
    "**注意**: 一般情况下论文使用的是随机划分的数据，但也有些论文使用【刻意】筛选的数据划分。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9ee4c308",
   "metadata": {},
   "outputs": [],
   "source": [
    "targets = []\n",
    "for l in labels:\n",
    "    new_models = okcomp.comp1.create_clf_model(model_names).values()\n",
    "    for m in new_models:\n",
    "        m.fit(X_train_sel, y_train_sel[l])\n",
    "        y_pred = m.predict(X_test_sel)\n",
    "    targets.append(new_models)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f97bb6e",
   "metadata": {},
   "source": [
    "## 预测结果\n",
    "\n",
    "* predictions，二维数据，每个label对应的每个模型的预测结果。\n",
    "* pred_scores，二维数据，每个label对应的每个模型的预测概率值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "792ab9e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score, roc_curve, auc\n",
    "from sklearn.preprocessing import OneHotEncoder\n",
    "\n",
    "predictions = [[model.predict(X_test_sel) for model in target] for label, target in zip(labels, targets)]\n",
    "pred_scores = [[model.predict_proba(X_test_sel) for model in target] for label, target in zip(labels, targets)]\n",
    "\n",
    "metric = []\n",
    "for label, prediction, scores in zip(labels, predictions, pred_scores):\n",
    "    for mname, pred, score in zip(model_names, prediction, scores):\n",
    "        enc = OneHotEncoder(handle_unknown='ignore')\n",
    "        y_test_binary = enc.fit_transform(np.array(y_test_sel[label]).reshape(-1, 1)).toarray()\n",
    "        fpr, tpr, _ = roc_curve(y_test_binary.ravel(), score.ravel())\n",
    "        metric.append((mname, accuracy_score(y_test_sel[label], pred), auc(fpr, tpr), label))\n",
    "        \n",
    "metric = pd.DataFrame(metric, index=None, columns=['model_name', 'Accuracy', 'AUC', 'Task'])\n",
    "metric"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0720a2cd",
   "metadata": {},
   "source": [
    "### 绘制曲线\n",
    "\n",
    "绘制的不同模型的准确率柱状图和折线图曲线。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b2f8cd6",
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "sns.set_theme(style=\"whitegrid\")\n",
    "\n",
    "plt.figure(figsize=(10, 10))\n",
    "plt.subplot(211)\n",
    "sns.barplot(x='model_name', y='Accuracy', data=metric, hue='Task')\n",
    "plt.subplot(212)\n",
    "sns.lineplot(x='model_name', y='Accuracy', data=metric, hue='Task')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0555dd2e",
   "metadata": {},
   "source": [
    "## 绘制ROC曲线\n",
    "确定最好的模型，并且绘制曲线。\n",
    "\n",
    "```python\n",
    "def draw_roc(y_test, y_score, title='ROC', labels=None):\n",
    "```\n",
    "\n",
    "`sel_model = ['SVM', 'KNN']`参数为想要绘制的模型对应的参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f6efdd6",
   "metadata": {},
   "outputs": [],
   "source": [
    "sel_model = ['SVM', 'KNN']\n",
    "\n",
    "for sm in sel_model:\n",
    "    if sm in model_names:\n",
    "        sel_model_idx = model_names.index(sm)\n",
    "    \n",
    "        # Plot all ROC curves\n",
    "        plt.figure(figsize=(8, 8))\n",
    "        okcomp.comp1.draw_roc([np.array(y_test_sel[l]) for l in labels], \n",
    "                              [s[sel_model_idx] for s in pred_scores], labels=labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "15bc513a",
   "metadata": {},
   "source": [
    "## 绘制混淆矩阵\n",
    "\n",
    "绘制混淆矩阵，[混淆矩阵解释](https://baike.baidu.com/item/%E6%B7%B7%E6%B7%86%E7%9F%A9%E9%98%B5/10087822?fr=aladdin)\n",
    "`sel_model = ['SVM', 'KNN']`参数为想要绘制的模型对应的参数。\n",
    "\n",
    "如果需要修改标签到名称的映射，修改`class_mapping={1:'1', 0:'0'}`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99ee12a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 设置绘制参数\n",
    "sel_model = ['SVM', 'KNN', 'XGBoost', 'LightGBM']\n",
    "c_matrix = {}\n",
    "\n",
    "for sm in sel_model:\n",
    "    if sm in model_names:\n",
    "        sel_model_idx = model_names.index(sm)\n",
    "        for idx, label in enumerate(labels):\n",
    "            cm = okcomp.comp1.calc_confusion_matrix(predictions[idx][sel_model_idx], y_test_sel[label],\n",
    "                                                    class_mapping={1:'1', 0:'0'}, num_classes=2)\n",
    "            c_matrix[label] = cm\n",
    "            plt.figure(figsize=(5, 4))\n",
    "            plt.title(f'Model:{sm}')\n",
    "            okcomp.comp1.draw_matrix(cm, norm=True, annot=True, cmap='Blues')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d24e2b7f",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
